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ABSTRACT 

This paper examines the Merck Institute for Science Education 
(MISE) . Begun in 1993, MISE had two stated goals: raise the interest, 
participation, and performance of public-school students in science; and 
demonstrate to other businesses that direct, focused involvement would hasten 
improvements in science teaching and learning in public schools. MISE formed 
partnerships with four school districts in New Jersey and Pennsylvania and 
evolved through four phases, each of which reflects the program’s approach to 
reform. The first phase centered on the development of a shared vision. This 
entailed the cultivation of relationships with each district by engaging 
education leaders in open discussions about science standards, the quality of 
the existing science curriculum, and expectations for their students. Phase 2 
was dedicated to building cultures of. instructional improvement at the 
district schools. The next phase featured the broadening of access to 
professional development and involved intense efforts to convince teachers to 
participate in peer-teacher workshops. Phase 4 consisted of sustaining the 
work and solidifying the peer-teacher network. The report outlines MISE's 
overall approach to professional development, the impact of MISE’s work, and 
the measurement of student performance under this program. It concludes that 
this systemic approach works and that sustained support for reform changes 
the norms of practice. (Contains 13 references.) (RJM) 
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The Evolution of the Merck Institute for Science Education 



Year 


Primary Focus 


Major Accomplishments 


1992-1993 


Launching the initiative 


Merck & Co., Inc. created the Merck Institute for 
Science Education (MISE) with a 10-year, $20-million 
commitment and the goal of raising student interest, 
participation, and performance in science. 


1993-1994 


Building awareness and providing assistance, and 
setting the groundwork for use of nationally 
developed curriculum materials 


The newly created MISE focused on cultivating 
relationships with its four partner districts, three in 
New Jersey and one in Pennsylvania. MISE assisted 
districts with selection and purchase of new 
materials for elementary science; and helped local 
educators envision a new approach to science 
education by sponsoring their attendance at 
national conferences, exposing them to state-of-the- 
art materials and national resources, and 
encouraging them to visit classrooms with 
standards-based science instruction. MISE created 
resource center enabling educators to review and try 
out new instructional materials. 


1994-1995 


Taking a more proactive role, evolving into a single 
Partnership, and designing and implementing the 
Leader Teacher Institute 


MISE focused on improving the quality and 
accessibility of professional development for 
teachers. MISE staff assumed a more proactive 
leadership role and, in the summer of 1995, 
implemented the Leader Teacher Institute, enrolling 
more than 140 teachers. This offered a common 
professional development experience for teams of 
teachers in the four districts. This helped create a 
single Partnership - MISE and the four districts. 


1995-1996 


Increasing resources through a National Science 
Foundation grant, implementing the Peer Teacher 
Workshops, continuing the Leader Teacher Institute, 
and focusing on local policy alignment 


MISE received a National Science Foundation Local 
Systemic Change grant for the purpose of providing 
100 hours of high-quality professional development 
in science and math to 800 K-8 teachers from the 
four districts over five years. The second year of the 
Leader Teacher Institute was held. Peer Teacher 
Workshops were implemented and more than 160 
teachers participated in the first summer. As Leader 
Teacher teams assumed more professional 
development roles, MISE staff evolved from a 
supplier of professional development to a facilitator 
of schoolwide instructional change. 


1996-1997 


Continuing the Peer Teacher Workshops, completing 
the Leader Teacher Institute, and focusing on 
curriculum frameworks and assessment 


Two hundred teachers took part in the Peer Teacher 
Workshops, which were partially led by Leader 
Teachers. The final year of the Leader Teacher 
Institute was held. MISE gave more attention to 
mathematics. The MISE Resource Center was 
expanded to include material for elementary and 
middle school math. All four districts completed 
draft science curriculum frameworks aligned with 
state and national standards. 


1997-1998 


Increasing district responsibility for professional 
development, expanding assessment work, initiating 
comprehensive planning, and continuing work with 
Leader Teachers 


The Partnership had evolved into a broad 
collaboration. Peer Teacher Workshops were 
expanded with 138 teachers participating. 
Communication and leadership skills of Leader 
Teachers were expanded to support them as 
advocates, coaches, and instructors in their schools. 
MISE staff worked to gain board approval for district 
curriculum frameworks. Work on improving student 
achievement measures started. 
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Year 


Primary Focus 


Major Accomplishments 


1998-1999 


Expanding professional development offerings, 
strategic planning in science, developing a 
Partnership Assessment Plan, sharing the work of 
the Partnership, influencing New Jersey policy 


MISE increased professional development offerings 
In the summer of 1999 and helped districts organize 
and deliver 36 Peer Teacher Workshops, thus 
building internal district capacity. MISE staff worked 
with district teams to develop strategic plans that 
focused on curriculum and instruction, student 
achievement and participation, policies and 
practices, and parent and community support. The 
Partnership adopted an action plan for student 
assessment in science. MISE staff expanded 
outreach efforts with new publications that outlined 
their vision and work. An assessment sampler for 
teachers was developed. MISE staff provided 
leadership to statewide boards and committees 
developing science content standards and 
professional teaching standards. 


1999-2000 


Continuing the Peer Teacher Workshops, expanding 
and focusing the work in the middle schools, 
implementing the Partnership's assessment plan 


MISE and the districts offered 31 Peer Teacher 
Workshops in science and math in the summer of 
2000, held in district locations to increase 
participation. MISE staff expanded and strengthened 
work in middle schools. Teams of teachers and MISE 
staff selected, modified, and tested two TIMSS (Third 
International Mathematics and Science Study) tasks 
for third and seventh grades. 


2000-2001 


Expanding the district role in the design and delivery 
of the Peer Teacher Workshops, implementing the 
Principal's Institute, working on site in three middle 
schools, and working on the Performance 
Assessment project 


MISE continued to offer Peer Teacher Workshops. A 
two-day institute for school principals was held to 
increase their understanding of high-quality science 
instruction and their capacity to help teachers 
provide it. MISE staff continued to work on the 
development of curriculum frameworks and the 
selection of instructional materials for the middle 
grade level. The Partnership Performance 
Assessment project was replicated in all grade 3 and 
7 classrooms. 


2001-2002 


Continuing the enhancement of local district 
responsibilities for Peer Teacher Workshops, 
implementing the Principal's Institute, working on 
site in three middle schools, initiating work at one 
partner high school, and enhancing the 
Performance Assessment project 


Peer Teacher Workshops reflected the needs of a 
context where there is a high level of district 
commitment to standards-based science. A two-day 
conference continued the institute for principals to 
focus on their roles in encouraging good science 
teaching through teacher observations. MISE staff 
worked with district committees to establish formal 
curriculum frameworks for school board adoption. 
Science reform work was initiated at one partner 
high school using the selection of instructional 
materials as the reform focus. The Partnership 
Performance Assessment project was expanded with 
tasks administered at grades 3, 7, and 8. 


2002-2003 


Continuing Peer Teacher Workshops, continuing the 
Principal's Institute, working on site with middle 
schools, implementing science reform work at one 
partner high school, and initiating district-level 
assessments for science modules 


Peer Teacher Workshops responded to the needs of 
districts committed to standards-based science. A 
second conference for principals focused on the 
power of professional dialogue about instruction. 
Science reform work continued at one partner high 
school using the selection of instructional materials 
as the reform focus. Two partner districts cooperated 
to develop, administer, and interpret summative 
assessments aligned with instructional modules. 
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Introduction 

A variety of technical assistance 
organizations have been created in the 
last 20 years to help public schools imple- 
ment reforms to improve their perfor- 
mance. These organizations vary in size, 
sponsorship, and focus, but their creation 
rests on the common premise that the 
reforms needed in the schools to educate 
all children to high standards require 
strong external stimuli and resources and 
knowledge beyond what are ordinarily 
available in public school systems 
(McDonald, McLaughlin, & Corcoran, 
2000). Accordingly, these technical assis- 
tance organizations forge partnerships 
with school systems under pressure to 
improve their performance. Working 
across the boundaries of the educational 
system, these organizations serve as 
catalysts for reform, offering schools and 
districts expertise and other resources 
needed to make the desired changes. 
Dedicated to the implementation of 
reforms, they are presumed to be free of 
the ordinary interests and ordinary 
political pressures and, therefore, more 
likely to be able to overcome the inertia 
and resistance that often block reform in 
public bureaucracies like school systems. 

Researchers have not paid sufficient 
attention to these organizations, yet they 
play an increasingly important role in the 
improvement of public education. To 
stimulate more interest in these organiza- 
tions, and in understanding what makes 
them effective, we report here on the 
Merck Institute for Science Education 
(MISE). For nearly 10 years, the Consor- 
tium for Policy Research in Education 
(CPRE) has evaluated MISE's partnership 
with four school districts in New Jersey 
and Pennsylvania, providing MISE staff 
with feedback on the progress of their 
work and assessing MISE's impact on 
schools, teachers, and students. This 
long-term relationship has provided an 
extraordinary opportunity for both CPRE 



and MISE staff to gain insights into how a 
technical assistance organization works 
with school districts to change classroom 
practice. The story of MISE, and its efforts 
to bring about instructional reforms in 
science, is a story of vision, collaboration, 
learning, and persistence. It is also a 
success story that offers important lessons 
for other intermediary organizations 
working with school districts to improve 
teaching and learning. 

Origins of MISE 

Merck & Co., Inc., the third largest 
pharmaceutical company in the world in 
terms of revenues (College Journal, n.d.), 
has a long history of supporting public 
education. Before school-business part- 
nerships became popular, Merck was 
contributing grants, gifts, and the time 
and talents of its employees to the im- 
provement of science education in public 
schools and higher education. However, 
Merck executives were not entirely 
satisfied with these efforts to assist the 
public schools. They knew that although 
their gifts were appreciated by the com- 
munities that received them, their impact 
on teaching and learning in the public 
schools was limited. They were also 
concerned about the declining supply of 
well-trained scientists and technicians in 
the United States and felt that steps had 
to be taken to increase the interest of 
American students in science. And as a 
pharmaceutical company, they also 
recognized that new developments in 
science, such as biotechnology, required a 
higher level of scientific literacy in the 
general population. 

After participating in a national 
education summit convened by President 
Bush and the nation's governors in 1991, 
at which national goals for the improve- 
ment of education in the United States 
were established for the first time, Merck 
executives felt that the corporation 
should make a significant and visible 
commitment to improving science educa- 
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The Study of MISE 

Since 1993, CPRE has been docu- 
menting MISE's work in the four part- 
ner districts, and providing MISE with 
feedback on the implementation of its 
strategies and their impact. Between 
1993 and 2002, CPRE regularly inter- 
viewed school and district staff, ob- 
served science classrooms, surveyed 
teachers, developed case studies of 
schools, examined student achievement 
data, and conducted special analyses to 
answer questions of concern to MISE 
and the districts. The long-term charac- 
ter of this research and the breadth of 
the data set provide a unique look at the 
development and impact of a technical 
assistance organization. This work is 
reported in eight annual reports, pub- 
lished between 1994 and 2002, which 
are available from CPRE. 

tion. Although becoming first in the 
world in mathematics and science by the 
year 2000 seemed unrealistic, they be- 
lieved significant progress was possible 
with a more focused and hands-on 
approach. In 1993, Merck created the 
Merck Institute for Science Education and 
supported the new venture with a 10- 
year, $20-million financial commitment. 
From its inception, MISE had two goals: 
to raise the interest, participation, and 
performance of public school students in 
science, and to demonstrate to other 
businesses that direct, focused involve- 
ment would hasten the improvement of 
science teaching and learning in the 
public schools. 

MISE quickly formed partnerships 
with four school districts in which Merck 
facilities were located and where area 
residents were familiar with a Merck 
presence in their schools. Three of these 
districts were in New Jersey — Linden, 
Rahway, and Readington Township — 
and the fourth was the North Penn 
School District in Pennsylvania. 



MISE invited leadership teams from 
each district — including the district 
superintendent, the science and/ or 
mathematics supervisor, school princi- 
pals, and Leader Teachers — to serve on a 
district advisory committee and to pro- 
vide input on critical decisions affecting 
its work. Over time, this advisory com- 
mittee developed into a learning commu- 
nity that shared a common reform 
agenda, and the individual partnerships 
between MISE and the four school dis- 
tricts evolved into one unified Partner- 
ship. This enabled MISE and the four 
districts to work collaboratively as one 
entity, united in purpose and goals. 

Taxpayers and parents in the four 
districts were delighted to have their 
school systems form partnerships with a 
corporate neighbor whose expertise in 
science was respected and whose largesse 
had been bestowed on their schools in the 
past. This time, however, the relationship 
would be different, not only because of 
the increased investment by Merck, but 
also because the new activist strategy 
represented by MISE sought greater 
commitments from the school districts; 
changes in district policies, culture, and 
organization; and specific reforms in 
curriculum and instruction. 

A New Vision for Science 
Education 

Guided by a board of advisors that 
included representatives of the National 
Science Resources Center, the National 
Science Teachers Association, the Na- 
tional Academy of Sciences, and leading 
scientists and science educators, and led 
by a respected science educator, a small 
MISE staff set out to reform science 
teaching in the elementary and middle 
grades in the four school districts. They 
sought a dramatic transformation from 
textbook-based, memorization-oriented 
instruction to guided inquiry in which 
students actively engaged in science 
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investigations based on structured cur- 
riculum units such as those developed by 
FOSS (Full Option Science System) and 
STC (Science and Technology for Chil- 
dren). While some reformers subscribe to 
a more radical view of inquiry in which 
students determine what topics and 
questions they wish to explore and 
design and conduct their own inquiries, 
MISE offered a more practical vision 
suited to K-8 teachers. Their goals were to 
persuade districts to adopt well-designed, 
commercially available science modules 
that would support inquiry or investiga- 
tions guided by teachers and cover the 
key concepts identified in state and 
national standards, and to help districts 
prepare teachers to use these instructional 
materials both effectively and consis- 
tently. 

While supportive of student inquiry, 
MISE staff recognized that teachers and 
schools needed a more structured ap- 
proach to be successful. This strategy was 
termed inquiry-centered science teaching. 
MISE wanted inquiry-centered teaching 
to be the norm in science classrooms, an 
integral and regular part of the experi- 
ence of all students at all grade levels. In 
these classrooms, students would be 
developing and exploring scientifically 
oriented questions, giving priority to 
evidence, and formulating, evaluating, 
and communicating their explanations. 
This vision of high-quality science educa- 
tion was new to many administrators and 
teachers in the partner districts. And 
moving elementary science teaching 
away from a heavy dependence on 
textbooks and toward inquiry posed 
significant challenges. As in most school 
districts across the United States, science 
was being taught by generalist teachers 
whose science backgrounds were often 
minimal — typically consisting of only 
one or two basic college courses. In fact, 
making sure science was taught at all in 
the elementary grades was an issue. 

There were no state assessments in 
science when the MISE staff began their 



work, and the reputations of elementary 
schools and their relationships with the 
state departments in New Jersey and 
Pennsylvania depended on the perfor- 
mance of students in reading, writing, 
and mathematics, not science. Thus, in 
many elementary classrooms, little 
attention was given to science, and when 
it was taught, it was often merely an 
extension of the reading program. 

To put this new vision of science 
teaching into practice, the MISE staff 
realized that they would have to do more 
than provide professional development 
for teachers. Their strategy was systemic 
and was based on studies of previous 
efforts to improve teaching (Corcoran & 
Goertz, 1995; Goertz, Floden, & O'Day, 
1995). The theory included the following 
nine components: 

• Persuading districts to make the 
improvement of science teaching a 
priority, and to engage in serious 
planning to address it; 

• Developing a leadership team in the 
district that shared a common vision 
of science teaching grounded in 
inquiry and consistent with state and 
national standards; 

• Helping districts develop new cur- 
riculum frameworks for science, select 
appropriate instructional materials, 
and develop systems for the manage- 
ment of the materials to ensure they 
could be used effectively; 

• Supporting the use of assessments 
that were consistent with the vision of 
good teaching, including formative 
assessments, end-of-unit assessments, 
and district-wide performance assess- 
ments and examinations; 

• Building district capacity to plan and 
deliver professional development that 
prepared teachers to use the materials 
effectively; 
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• Developing district and school in- 
structional leaders to carry out this 
work; 

• Developing professional cultures for 
the districts and schools that would 
promote continuous improvement of 
science teaching and develop teacher 
expertise; 

• Aligning district policies for curricu- 
lum, professional development, 
resource allocation, and teacher 
evaluation with the vision of re- 
formed practice and the strategies for 
improvement; and 

• Promoting supportive state policies. 

The leaders of MISE felt that only 
such a systemic approach could stimu- 
late, support, and sustain the reforms in 
classroom practice that they sought. They 
realized that new curriculum materials 
and professional development would be 
needed, but they also recognized that 
school and district policies governing 
curriculum, assessment, professional 
development, resource allocation, person- 
nel evaluation, and accountability would 
have to be aligned with the new vision of 
high-quality science teaching. 

For nearly a decade, MISE has pur- 
sued this vision and CPRE has been 
documenting their work and its impact. 
What follows is the story of MISE, and 
the lessons that have been learned from 
its efforts to improve science teaching. 

Learning from Experience 

The story of MISE can be divided 
roughly into four phases reflecting the 
elaboration of MISE's approach to reform. 
In each phase, MISE altered its strategies 
as a result of feedback from its partners 
and CPRE, changes in the policy environ- 
ment, and new funding opportunities. 



Phase 1: Developing a Shared 
Vision 

In 1993, neither MISE's partner 
districts nor the two states (New Jersey 
and Pennsylvania) had content standards 
or assessments in science. District cur- 
riculum guidelines for science were often 
vague and often ignored. Most teachers 
lacked the materials and equipment 
necessary to engage in inquiry. Instruc- 
tional expenditures for science materials 
were low. Shaped by teacher interests and 
accountability demands on the schools, 
science instruction was often little more 
than an extension of the reading curricu- 
lum. In some classrooms, children ex- 
plored central concepts in science, but in 
many others, they did little more than 
read about nature. This was the condition 
of science education that MISE, in part- 
nership with the districts, set out to 
change. 

In the first two years, MISE staff set 
out to build a shared vision of good 
science teaching with the partner dis- 
tricts. They cultivated relationships of 
trust with each district by engaging 
district superintendents, central office 
staff, principals, and teacher leaders in 
open discussions about the science 
standards, the quality of the existing 
science curriculum, and their expecta- 
tions for their students. They convinced 
the districts to re-examine and revise 
their K-8 science curricula. Leadership 
teams from each district went to the 
National Science Resources Center's 
Elementary Science Leadership institutes 
with technical and financial support from 
MISE. Each attending team worked on a 
five-year strategic plan for reforming 
science education in their system. The 
institutes focused on the selection of 
inquiry-centered sequential science 
curriculum units, professional develop- 
ment to prepare teachers to use inquiry- 
centered teaching methods, cost-effective 
support systems for supplying science 
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materials and apparatus to classrooms, 
assessment methods consistent with 
inquiry-centered science, and strategies 
for building administrative and commu- 
nity support. Upon their return, MISE 
staff worked with them to review the 
science modules that were available for 
elementary- and middle-grade science, 
and assess their suitability for inquiry 
teaching. Through this process, a shared 
vision of inquiry-centered science teach- 
ing emerged. 

Rather than adopting new curricula 
all at once, MISE encouraged the districts 
to select high-quality science modules for 
each grade as the foundation of their 
elementary science curricula and to phase 
them in gradually so as not to overwhelm 
teachers. The modules selected by the 
districts fit the shared vision of inquiry 
teaching. To support this process, MISE 
opened a resource center that loaned out 
instructional materials so that district 
staff could examine those that fit what 
they were trying to do in the classroom. 
This allowed local teachers to test out the 
new science modules in local schools 
prior to their purchase and adoption. 
Once districts selected appropriate mate- 
rials, MISE funded the purchase of these 
new curricular modules which became 
the foundation of a more rigorous, stan- 
dards-based elementary science curricu- 
lum in each of the partner districts. In 
addition, MISE encouraged partner 
districts to examine their personnel 
policies and procedures to ensure they 
were recruiting and hiring qualified 
teachers and to develop strategic plans 
that would incorporate the instructional 
reforms sought by the Partnership. In the 
first two or three years of the Partnership, 
all four districts made major changes in 
their elementary science curriculum. 



Phase 2: Building Cultures of 
Instructional Improvement 

The adoption of new instructional 
materials in the elementary schools 
quickly generated requests from teachers 
for more and better professional develop- 
ment and on-site support. Teachers 
needed better understanding of the 
subject matter addressed by the new 
curriculum materials so they could 
encourage students to ask critical ques- 
tions and help them seek meaningful 
answers. They also needed to understand 
how to organize opportunities for in- 
quiry, how to set up their classrooms, and 
how to replenish supplies for and man- 
age the new science modules. Building 
supports for instructional change became 
the focus of the next few years as MISE 
continued to support the adoption of the 
new curriculum materials. 

With the assistance of the partner 
districts, MISE designed and imple- 
mented a new program to prepare Leader 
Teachers in each school to support the use 
of the new curriculum and the use of 
inquiry in the classroom. Between 1995 
and 1998, the partners designed and 
implemented a voluntary three-year 
professional development experience for 
teams of teachers selected from each 
elementary and middle school in the four 
partner districts. This Leader Teacher 
Institute (LTI) provided three-week 
summer institutes and academic-year 
sessions focused on a different domain of 
science (biological, physical, and earth 
sciences) each year intended to deepen 
teachers' content knowledge, sharpen 
their teaching skills, and prepare them to 
serve as mentors, coaches, and advocates. 
The creation of the LTI had an important 
impact on the partnerships themselves. 

By recruiting more than 140 teachers from 
all four districts into a common profes- 
sional development experience, the four 
separate partnerships became one unified 
Partnership. The LTI required significant 
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commitment from MISE, the districts, and 
the teachers, and these commitments 
forged a bond that would prove lasting. 

Impact on Teaching Practices . The 
Leader Teachers received nearly 500 
hours of professional development over 
three years, and observations of their 
practice by CPRE researchers found that 
their teaching changed dramatically by 
the third year and was considerably 
above average on a National Science 
Foundation (NSF) observation classroom 
instrument. Survey data also showed 
significant changes in their practice. By 
the third year, all of the Leader Teachers 
had made significant changes in their 
practice and the vast majority were 
engaging students in inquiry on a regular 
basis. After three years of involvement, 
however, their practice began to stabilize 
as they encountered the time limits faced 
by all elementary teachers who teach 
multiple subjects and pressures to attend 
more to reading and mathematics. 

Spreading Reformed Practice . The 
goal of the LTI was to increase the inter- 
nal capacity of each school in the partner 
districts to support change in instruc- 
tional practice. The Leader Teachers were 
expected to serve as advocates of inquiry- 
centered instruction, and to coach other 
teachers in these methods. Many of them 
conducted professional development for 
their peers, and some went into class- 
rooms to observe and coach. After two 
years, however, it was clear that the 
impact of this change strategy was un- 
even. Some Leader Teachers were suc- 
cessfully engaging their colleagues but 
others encountered serious obstacles, 
such as their own credibility as science 
teachers, time constraints, peer resent- 
ment, and lack of principal support. 
(Detailed accounts of the strategies used 
by Leader Teachers and the obstacles they 
encountered can be found in CPRE, 1999, 
pp. 56-63, and CPRE, 2000, pp. 59-64). 



The lessons from the LTI initiative 
were not lost on MISE. Successful diffu- 
sion of reformed practice in the schools 
clearly depended on which individuals 
were selected to be Leader Teachers; the 
individuals not only had to have interest 
in, and knowledge of, science but they 
also had to be respected by other teachers 
in the school and have good relationships 
with their colleagues. The LTI selection 
process had not always resulted in the 
recruitment of those with the most poten- 
tial for effectiveness. And once selected, 
the Leader Teachers had to have the 
support of principals. To succeed in 
reaching their peers, they also needed 
released time that districts and principals, 
facing demands to improve reading and 
mathematics, sometimes would not 
provide. Given these obstacles and the 
prevailing norms of high teacher au- 
tonomy and classroom privacy, spreading 
inquiry through the Leader Teachers 
required a cultural revolution in each 
school that was difficult to spark. 

But the LTI experience offered another 
powerful lesson for MISE and its partner 
districts. Intensive professional develop- 
ment had altered the practice of the 
Leader Teachers, and the Leader Teachers 
felt that similar opportunities would 
encourage their fellow classroom teachers 
to adopt the inquiry method and help 
them use the new curriculum materials 
effectively. Although MISE all along had 
been encouraging district efforts to 
provide all elementary teachers with 
more professional development in sci- 
ence, the announcement of the Local 
Systemic Change program by NSF pro- 
vided the Partnership with the opportu- 
nity to expand and intensify these learn- 
ing opportunities. 

Phase 3: Broadening Access 
to Professional Development 

In 1995, the Partnership received a 
five-year Local Systemic Change grant 
from NSF to provide more teachers with 
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Table 1. Peer Teacher Workshops, 1996-2001 



Year 


Number of 
PTWs 


Math 

PTWs 


Math and 
Science 
PTWs 


Science 

PTWs 


PTW 

Enrollment 


Number of 
Individuals 


Summer 1996 


6 


0 


0 


6 


169 


169 


Summer 1997 


8 


2 


0 


6 


195 


195 


Summer 1998 


22 


8 


1 


13 


506 


287 


Summer 1999 


36 


9 


1 


26 


525 


386 


Summer 2000 


32 


9 


2 


21 


667 


490 


Summer 2001 


37 


12 


1 


24 


536* 


394* 


Total 


141 


40 


5 


96 


2,598 


1,921 



* includes estimated enrollments in locally sponsored math PTWs. 
Source: CPRE (2002, p. 19) 



direct access to curriculum-related profes- 
sional development in both science and 
mathematics. The centerpiece of this 
project was a new professional develop- 
ment strategy, the Peer Teacher Workshop 
(PTW), that would make intensive profes- 
sional development available to all 
teachers. Each week-long PTW was based 
on one of the curriculum modules used in 
the districts, and was designed to deepen 
teachers' understanding of the science in 
the module, their appreciation of com- 
mon student misunderstandings of the 
key concepts, and their ability to use 
inquiry. The encouragement of teachers to 
attend as grade-level teams, the organiza- 
tion of follow-up sessions during the 
school year, and support from district 
resource staff meant that the PTWs were 
also excellent vehicles for building and 
strengthening professional learning 
communities in the schools that could 
carry and deepen the vision of inquiry- 
based science. The goal was to provide at 
least 80% of the 800 teachers who taught 
math and science in grades K-8 in the 
four partner districts with 100 hours of 
this kind of professional development 
over a five-year period. 



The PTWs were designed by instruc- 
tional teams comprised of accomplished 
teachers from the partner districts and 
content experts drawn from MISE staff, 
high schools, universities, and curriculum 
development organizations. The new 
strategy utilized the local capacity devel- 
oped through the LTI by recruiting 
successful Leader Teachers to help design 
and deliver the PTWs. The three-to-four 
person instructional teams were selected 
by MISE and district staff and supported 
by an annual three-day design retreat 
held in the spring several months before 
the workshops. At these annual retreats, 
MISE staff and other experts helped the 
instructional teams design workshops 
that modeled inquiry, taught the partici- 
pants the underlying science or math- 
ematics, and prepared them to manage 
the curriculum unit. 

Table 1 displays the number, types, 
and enrollment of Peer Teacher Work- 
shops held since 1996. These data show 
steady growth in participation from 1996 
to 2000 and then a leveling off at a rela- 
tively high level of participation. During 
1999, 2000, and 2001, approximately half 
of the nearly 800 K-8 teachers teaching 
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science and/or mathematics in the four 
districts participated in PTWs. By 2002, 
over 80% had participated which, given 
the other demands on teachers' time 
during this period, especially the strong 
emphasis placed on reading, was a 
significant accomplishment. 

Improving Participation 

This high rate of participation was not 
so easily achieved. After the PTWs had 
been in place for three years, approxi- 
mately 40% of the teachers in the districts 
still had not participated. While observ- 
ing and interviewing teachers in the 
Partnership districts, CPRE noted that 
teachers who had not yet participated in 
Partnership professional development 
were often quite knowledgeable about 
MISE's involvement in their district and 
some described the Partnership in glow- 
ing terms. CPRE initiated a study of these 
non-participants and interviewed a 
stratified (by grade level) random sample 
of approximately 10% of the teachers who 
had not yet participated in the 
Partnership's professional development. 

Survey data revealed no differences 
between participants and non-partici- 
pants in terms of experience. Teachers 
with over 20 years of experience were 
equally as likely to participate in the 
PTWs as their younger or less-experi- 
enced peers. Furthermore, participants 
did not have stronger backgrounds in 
science. Differences between participants 
and non-participants were found in three 
crucial areas: their enjoyment of teaching 
science, their preparation to teach it, and 
their conceptions of professional develop- 
ment. Not surprisingly, participants 
reported greater enjoyment of teaching 
science and being better prepared to teach 
it. Not much could be done about these 
attitudinal differences unless non-partici- 
pants could be persuaded to attend the 
PTWs. 



The information about non-partici- 
pants' conceptions of effective and desir- 
able professional development was more 
useful. Their perceptions collided with 
the design of the PTWs in four important 
ways, described more fully below. First, 
the length of the PTW clashed with their 
expectations. Second, the professional 
development strategies were inconsistent 
with non-participants' preferences. Third, 
the content of the professional develop- 
ment was not a priority for them. Fourth, 
the timing and location of the PTWs were 
barriers to participation. Thus, the struc- 
ture, style, content, and logistics of the 
PTWs were perceived to be barriers to 
participation. 

Non-participating teachers over- 
whelmingly preferred short professional 
development experiences. Most of those 
interviewed said that committing a full 
week to professional development was 
extremely unattractive. They preferred 
one-day events that they could attend on 
professional release days provided by 
their districts. They were hesitant to 
commit more of their time outside of the 
school day to professional development. 

Non-participants also conceived of 
effective professional development as the 
dissemination of information, rather than 
more active exploration of larger ideas 
and concepts based on subject-content 
knowledge. These views conflicted with 
the philosophy behind the Partnership's 
professional development. When asked to 
describe the most effective structure of 
professional development, these teachers 
overwhelmingly preferred one-day 
workshops where they received materials 
and were led by an interesting instructor. 
Furthermore, the PTWs made the practice 
of many of the participants more public. 

It is important to recognize that the 
traditional culture of teaching is insu- 
lated; one's struggles and shortcomings 
are often kept behind closed classroom 
doors. Despite the supportive, reflective 
nature of these workshops, these teachers 
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were less willing to open their practices 
to their peers, and therefore they were 
less willing to attend the workshops. 

The content of the PTWs also con- 
flicted with non-participants' preferences. 
Most of the elementary teachers inter- 
viewed preferred professional develop- 
ment in the language arts rather than in 
science or math. Furthermore, the teach- 
ers interviewed were more likely to 
choose general areas for professional 
development than opportunities to 
deepen their content knowledge in any 
one area. Finally, offering the PTWs in the 
middle of the summer in locations that 
were some distance from teachers' homes 
also deterred the non-participants. They 
preferred professional development 
during the school year or immediately 
following the school year and preferred it 
to be offered in their districts. 

In response to these findings, the 
Partnership made changes to the struc- 
ture and marketing of its summer work- 
shops. The expanded offerings included 
some choices not directly tied to specific 
science curriculum units, such as using 
technology in the science classroom, 
assessment, and integrating science and 
language arts. These choices may have 
attracted teachers who preferred more 
general areas for professional develop- 
ment, who traditionally sought language 
arts workshops, or who needed to learn 
more about technology. Informal net- 
works of participating teachers also 
recruited their peers. The Leader Teachers 
were strongly encouraged to attend. In 
response to district requests, the Partner- 
ship changed the timing of the work- 
shops so they occurred immediately 
following the close of the school year. 
Also, one district was able to offer teach- 
ers the option of graduate credit. These 
efforts led to the recruitment during the 
following summer of 34% of those who 
had previously not participated, and 
eventually two-thirds of those who had 
not participated in the first three years 
(CPRE, 2000). 



Impact on Practice 

To analyze the impact of this profes- 
sional development on teaching practice 
as reported by teachers, CPRE evaluators 
examined the statistical relationship 
between reform-based teaching practice 
and Partnership-provided professional 
development while controlling for certain 
teacher background and school character- 
istics (CPRE, 1999). Hierarchical linear 
modeling, a sophisticated form of regres-. 
sion analysis that takes into account that 
classrooms are nested within schools, was 
used (Bryk & Raudenbush, 1992). 

Using data collected from surveys of 
teachers and principals in the four part- 
ner districts, CPRE evaluators aggregated 
a series of survey items about teaching 
practices in science. The survey items 
asked teachers how frequently they used 
certain practices, such as requiring stu- 
dents to supply evidence to support their 
claims, demonstrating a science-related 
principle or phenomenon, and using 
assessment to find what students know 
before or during a curriculum unit. 
Teacher content familiarity, teaching 
experience, and amount of professional 
development were used as independent 
variables to predict teaching practice. At 
the teacher level, the sample consisted of 
334 teachers. Twenty-eight percent of 
these teachers reported they had received 
no science-related professional develop- 
ment during the year prior to the study, 
33% reported receiving between 1 and 39 
hours, 17% reported receiving between 40 
and 79 hours, and 22% reported receiving 
80 or more hours. 

The most striking result from this 
model of science teaching is the statisti- 
cally strong relationship between high 
levels of quality professional develop- 
ment and reform-based teaching practice. 
After adjusting for differences in teachers' 
content background, teaching experience, 
and school environment, teachers with 
more than 79 hours of professional 
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Figure 1. Impact of Professional Development on 
Teacher-reported Science Teaching Practice 




development used significantly more 
reform-based teaching practices than did 
teachers with fewer than 79 hours of 
professional development. Figure 1 
depicts the relationship between teaching 
practice and professional development. 

Teachers who had either no profes- 
sional development or between 1 and 39 
hours of professional development had 
approximately average teaching practice 
(that is, their scores on the scales devel- 
oped from the survey items were at about 
the mean). Teachers who had between 40 
and 79 hours of professional develop- 
ment were slightly above average in 
terms of their use of reform-based teach- 
ing practice. Teachers who received 80 or 
more hours of professional development 
were much more likely to have altered 
their practice. 

Also notable was the relationship 
between content familiarity and reform- 
based teaching practice. Each additional 
semester of college science (a proxy for 
content familiarity) was associated with a 
statistically significant .11 of a standard 
deviation increase in the model's measure 
of reform-based teaching practice. This 
suggests that content-based professional 



development is important and bears 
further investigation. There was no 
significant relationship between years of 
teaching experience and reform-based 
practice. This finding casts doubt on the 
widespread assumption that experienced 
teachers are more reluctant to change 
their practice than relatively new teach- 
ers. 

Finally, direct observations of class- 
room practice by CPRE researchers in 
1998, 1999, and 2000 found both the 
Leader Teachers who had received inten- 
sive professional development in the 
second phase of MISE's work and the 
teachers who had attended the PTWs had 
changed their practice, and both were 
using the science modules to engage 
students in guided inquiry. Using an 
observation instrument developed by 
Horizon Research for the NSF Local 
Systemic Change evaluation, CPRE 
researchers found the Leader Teachers to 
be somewhat more advanced in their use 
of inquiry, but both groups were above 
average on the NSF scale, and the ob- 
served differences between them were 
narrowing each year (CPRE, 1999, pp. 34- 
36). 
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Phase 4: Sustaining the Work 

The Local Systemic Change grant 
ended in 2001, but the four districts, with 
modest support from MISE, have contin- 
ued to offer the PTWs. In the summer of 
2002, 25 PTWs were offered by the four 
districts. Moreover, the general design of 
the PTWs has been extended to other 
subject areas in all four districts. Each of 
the districts now has a cadre of teachers 
who have designed and led PTWs in 
science and mathematics. They have the 
capacity to offer teachers high-quality 
professional development. These experi- 
enced leaders also serve as coaches and 
mentors in their schools, and as curricu- 
lum leaders in their districts. The experi- 
ence serving on an instructional team 
gives them the knowledge, skills, and 
credibility to be leaders in other ways. 
Equally important, these experiences 
have raised the standards for professional 
development. Teachers in these districts 
expect well-designed practical experi- 
ences that enable them to do a better job 
with the curriculum that they must teach. 

MISE has continued its work in a 
number of noteworthy ways. For ex- 
ample, the Partnership offered a two-day 
institute for principals to enhance their 
understanding of high-quality science 
instruction and their capacity to help 
teachers provide it. Organized by princi- 
pals from the four partner districts and 
MISE and CPRE staff, the institute was 
attended by 41 principals and a number 
of central office staff. MISE staff also 
worked with districts on the develop- 
ment of curriculum frameworks and 
selection of instructional materials for 
middle school science. MISE continues to 
support and develop the work of the four 
partner districts and is expanding the 
Partnership to include other school 
districts. 



MISE’s Approach to 
Professional Development 

While MISE modified its approach to 
professional development as a result of 
experience and CPRE's evaluation, the 
core principles guiding its design were 
constant. According to these guiding 
principles, professional development 
should be: 

• Based on a clear vision of good prac- 
tice; 

• Linked to specific curriculum units 
and focused on the content teachers 
must teach; 

• Carefully designed and planned to 
provide knowledge and skills that 
were immediately useful in the 
classrooms of the participants; 

• Respectful of teachers and based on a 
coherent theory of adult learning; 

• Intensive but also extended over time 
through on-site support to allow for 
practice and reflection; 

• Led by accomplished teachers who 
modeled good instructional practice 
and collaborative work; 

• Easily accessible for all eligible teach- 
ers; and 

• Sustainable over time by local dis- 
tricts. 

These principles are consistent with the 
prevailing consensus on effective profes- 
sional development (Corcoran, 1995; 
Elmore, 2002; Sparks & Hirsh, 1997). 

MISE's use of the priorities of districts 
and schools to guide professional devel- 
opment differs from much current prac- 
tice which permits individual teachers to 
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choose their own learning opportunities. 
MISE's goal is to build a shared vision 
about sound instruction and communities 
of practice that can work together to enact 
it. While teacher participation in PTWs is 
voluntary, there are strong incentives for 
teachers to participate including peer 
pressure and stipends. Because there are 
no linkages between participation or 
performance in the PTWs and personnel 
evaluations, PTWs are seldom seen as 
high-stakes or threatening experiences. 
From the start, MISE sought to build 
district capacity to provide high-quality 
professional development on a sustained 
basis. Gradually, the responsibility for 
training has shifted to the partner dis- 
tricts where teachers and others trained 
through the PTWs are leading the work. 

Strengthening Content 
Knowledge 

As noted earlier, building teacher 
content knowledge in science has been a 
major focus of the Partnership's profes- 
sional development program. Needs of 
teachers were assessed through surveys, 
examination of student assessment 
results, and discussions with supervisors 
and principals. Most of the content 
experts used in the PTWs were skillful in 
addressing the content needs of partici- 
pants, linking activities and instructional 
ideas to the "big ideas" in the curriculum 
modules. Workshop participants re- 
viewed curriculum content and discussed 
problems that students were having with 
it. While CPRE researchers are persuaded 
that most PTW participants learned a 
great deal of new science content, there is 
only limited empirical evidence to sup- 
port this claim. CPRE's conclusion is 
based on observations of content discus- 
sions in the PTWs, the amount of time 
devoted to such discussions, the attention 
given to content in the workshop materi- 
als, and teacher self-reports about what 
they learned. 



The Systemic Context 

It is important to remember that this 
professional development was the center- 
piece of a broader reform effort, and that 
MISE's approach was systemic. They 
intentionally set out to alter the environ- 
ment in which teachers were working. In 
addition to helping the partner districts 
select new science curricula, MISE helped 
them develop new curriculum frame- 
works. Through the process of selecting 
materials and designing professional 
development, MISE had a profound effect 
on the norms of good practice in science 
teaching and the expectations of teachers 
held by central office curriculum staff. 
These new norms were also shared with 
principals through a series of institutes at 
which principals observed, rated, and 
discussed videos of science teaching. 
These other activities created environ- 
ments which were supportive of the 
professional development. 

The Impact of MISE’s 
Work 

The Partnership formed by MISE and 
the four school districts can point to some 
real accomplishments. Clearly, the science 
curriculum in the four partner districts 
has been strengthened and science has 
become a core subject in the elementary 
schools. MISE and district staffs have 
worked together to develop curriculum 
frameworks aligned to state and national 
standards, selected appropriate instruc- 
tional materials, and designed and devel- 
oped professional development activities 
that support the new curricula. The once 
fragmented, textbook-driven, and uncer- 
tain science curriculum has been replaced 
by well-structured curricula incorporat- 
ing well-designed science modules 
focused on important concepts linked to 
the state standards. Teachers have access 
to science modules and materials that 
were not available 10 years ago and, most 
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importantly, they are expected to use 
them. Students in the partner districts are 
now assured of exposure to a well- 
balanced, inquiry-centered science cur- 
riculum. 

Students also are assured of a better 
and more equitable opportunity to learn 
science because science instruction has 
also improved. Many teachers have a 
deeper understanding of the content and 
have adopted inquiry approaches as a 
result of the intensive professional devel- 
opment offered by the Partnership and 
support from teacher leaders and super- 
visors. Using NSF-developed observation 
tools that focus on the use of inquiry, 
CPRE has found the vast majority of 
elementary teachers in the Partnership 
schools to be using inquiry as a primary 
method for teaching science. 

District policies have also changed. 
The most obvious changes are the new 
curriculum frameworks, increased expen- 
ditures for instructional materials, obser- 
vation procedures, and the acceptance of 
MISE's principles of professional devel- 
opment. When the NSF grant ended and 
the amount of external funding was 
reduced in the summer of 2001, the 
districts continued to offer PTWs and still 
do so. They do so because there is both a 
need (new teachers, new modules, and a 
continued desire to do it better) and a 
demand (teachers want the kind of 
professional development provided 
through the PTWs). An unexpected by- 
product has been the use of the same 
professional development principles to 
support curriculum revision innovations 
in literacy, mathematics, and social 
studies. 

Other more subtle cultural changes 
have also occurred in the partner dis- 
tricts. There is more respect for, and 
greater use of, teacher expertise. Class- 
room teachers are playing leadership 
roles in professional development, assess- 
ment, and curriculum. Planning at the 



school and district levels is more collabo- 
rative and taken more seriously. There is 
a greater focus on results. Instructional 
resources have been reallocated to pro- 
vide teachers with more professional 
development opportunities and access to 
exemplary K-8 science education materi- 
als at MISE's resource centers. The re- 
sources devoted to professional develop- 
ment are seen as investments that should 
be carefully designed and are expected to 
produce results. 

Measuring Student 
Performance: Partial Answers 
and Continuing Challenges 

Have these changes improved student 
knowledge of science or increased the 
numbers of students who pursue more 
challenging courses in high school and 
college? Do students in the Partnership 
districts do better work in science than in 
other comparable districts? Do they score 
better on common measures of science 
understanding? Unfortunately, the 
answers to these questions are somewhat 
ambiguous. As will be shown below, we 
have some evidence that the reforms have 
impacted student achievement, but it is 
limited and the identified effects are 
modest. 

The ambiguity of the answers to 
questions about the impact on students 
stems from the challenges the Partnership 
has encountered in science assessment. 
None of the available measures of science 
achievement have proved adequate to the 
task of measuring the effects of inquiry- 
centered teaching. When MISE began its 
work, there were no standard measures of 
achievement in science used in the part- 
ner districts. In 1994, MISE and CPRE 
recommended the adoption of the SAT-9 
science assessment, but the districts, 
sensitive to criticism of over-testing, 
would only agree to using the open- 
response items in grades 5 and 7 and did 
not administer the full test. These limited 
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data are the only longitudinal student 
achievement data available for all four 
districts. In addition, for three of the 
districts, CPRE has examined four years 
of results (1999 to 2002) on the New 
Jersey Elementary School Performance 
Assessment (ESPA) science test adminis- 
tered to fourth graders, but there will be 
no more data from this source as the test 
has been eliminated as the state responds 
to the requirements of the No Child Left 
Behind Act of 2001. 

SAT-9 Results 

Analysis of the results on the SAT-9 
open-response items administered to fifth 
and seventh graders have not shown a 
consistent pattern of district-wide im- 
provement in the partner districts. How- 
ever, studies examining changes by 
classrooms have found some effects. In 
1999, CPRE staff examined the cumula- 
tive effects on students in grades 5, 6, and 
7 of being in classrooms led by teachers 
with different amounts of Partnership 
professional development. This analysis 
built upon, but was slightly different 
than, the work of Sanders and Rivers 
(1996). The hypothesis was that students 
who had different sequences of teachers 
with different amounts of Partnership 
professional development experience 
would perform differently. For example, 
students who studied science for three 
consecutive years in the classrooms of 
Leader Teachers would perform differ- 
ently, on average, on the SAT-9 than 
would students who had three consecu- 
tive years of teachers who had had not 
participated in Partnership professional 
development. The logic is that participat- 
ing teachers would have altered their 
practice and would be using more power- 
ful strategies to engage students in the 
study of science to help students master 
key concepts. 

The results showed that fifth-grade 
students whose teachers had one or two 
years of Partnership professional devel- 



opment during the 1996-1998 period 
outperformed students whose teachers 
had no Partnership professional develop- 
ment in the same time period. However, 
there was no statistical difference be- 
tween students who spent two years with 
teachers who participated in Partnership 
professional development and students 
who spent one year with such teachers. 
The seventh-grade analysis shows that 
students taught for two and three years 
by Partnership-trained teachers per- 
formed similarly statistically. These 
students performed better than students 
who spent just one year with a Partner- 
ship-trained teacher. No difference was 
found between seventh-grade students 
who had one year with a trained teacher 
and students who had no Partnership- 
trained teachers. Finally, it should be 
noted that these differences, while statis- 
tically significant, were substantively 
small (CPRE, 2002, pp. 45-49). 

There have been serious problems 
with using the SAT-9 as a measure of 
effectiveness. To the teachers and stu- 
dents in the four districts, the SAT-9 
represents additional testing (it is referred 
to as the "Merck test" or the "CPRE test") 
and, because no stakes are associated 
with the test, it was not always taken 
seriously by students or teachers. 

ESPA Results 

Analysis of the results of the ESPA 
administered in fourth grade in the three 
New Jersey districts have not shown 
significant effects. While the three MISE 
partner districts have ranked at or near 
the top of groupings of similar districts 
since the science test was first adminis- 
tered in 1999, CPRE has not been able to 
obtain student-level data from the New 
Jersey Department of Education to con- 
duct analyses. It has also been suggested 
that there have been differences in the 
difficulty of the test from year to year. The 
New Jersey fourth-grade ESPA also poses 
other problems. Although the items are 
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aligned with state standards, it is not 
possible to determine how well they are 
aligned with the specific content of the 
science modules used by the Partnership 
districts in the tested grades because the 
state department does not release the 
items. 

Moreover, since the stakes are quite 
high, all New Jersey districts are moti- 
vated to perform well. There may be little 
apparent difference in the results ob- 
tained on this assessment through con- 
ventional test preparation, increased time 
spent on science, and inquiry-based 
instruction. Thus, better teaching and 
better student work may not necessarily 
lead in the short-run to discernible differ- 
ences in results on the ESPA. 

Assessment Issues 

The lack of persuasive evidence that 
student performance had improved 
presented a serious dilemma for the 
Partnership. On one hand, school admin- 
istrators, teachers, MISE staff, and CPRE 
staff were in agreement that student work 
and student interest in, and understand- 
ing of, science had changed for the better 
in most Partnership schools. On the other 
hand, the available measures of student 
achievement were not recording signifi- 
cant gains. The consensus view was that 
the assessments were flawed and were 
not sensitive to the changes occurring in 
student learning. Moreover, MISE and 
district staff felt that the availability of 
better formative assessments linked to the 
science modules and increased use of 
performance assessments would contrib- 
ute to further improvements in teaching. 

The Assessment Plan 

In response to these problems, MISE 
and the partner districts developed a 
four-part plan in 1998 to improve assess- 
ment in science. MISE convened several 
meetings on the topic of student assess- 
ment in science to develop a consensus 
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on the best way to assess the effects of the 
instructional reforms being promoted by 
the Partnership. Interest in developing a 
more comprehensive assessment plan 
was driven in part by district dissatisfac- 
tion with the SAT-9 and the desire for 
better information. A variety of constitu- 
encies — including MISE, the four part- 
ner school districts, classroom teachers, 
Merck, and CPRE — shared these con- 
cerns. There was general agreement 
among the meeting participants on the 
importance of using a nationally recog- 
nized, standardized measure. However, 
the participants also wanted to collect 
assessment information more closely 
linked to the module-based curriculum 
used in the districts. They wanted perfor- 
mance information that could be stan- 
dardized and aggregated across schools 
and that was reliable enough to provide 
the basis for programmatic conclusions. 

These conversations culminated in 
development of a four-component Part- 
nership assessment plan. The four com- 
ponents were: 

1. A nationally recognized, standardized 
test consisting of multiple-choice 
and/or open-ended items focusing on 
central themes in science; 

2. A set of tasks consisting of perfor- 
mance, multiple-choice, or open- 
ended items more closely tied to the 
districts' curricula and standardized 
in their administration; 

3. Pre-/post- or summative tasks spe- 
cific to each science module that 
could be administered district-wide; 
and 

4. Informal, unit-based assessments of a 
variety of different forms for class- 
room use. 

Component 1. Finding a standardized 
test in science that was acceptable to all 
four districts was a source of continued 
frustration. A substitute was needed for 
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the SAT-9 which several districts had 
decided not to continue to use. Two of the 
partner districts adopted the TerraNova 
science test as part of more comprehen- 
sive district-wide adoptions, but four 
years after the plan was developed, there 
was still no Partnership-wide standard- 
ized assessment in science. 

The lack of a common curriculum in 
the four districts poses a fundamental 
problem for summative assessment. Each 
of the four partner districts has selected 
somewhat different science modules 
containing different content in the tested 
grades, so no common assessment can be 
well-aligned with each district's curricu- 
lum. Moreover, most of the four districts 
chose to rotate the use of the modules 
among classrooms rather than purchase 
them for each classroom. That means that 
the sequence in which the modules have 
been used has varied across classrooms 
within a school and among schools. As a 
consequence, it is impossible to achieve 
high content alignment within a single 
school with any test administered in the 
spring because classes will vary in the 
modules that they have completed. This 
presents a challenge for performance 
assessments as well as for standardized 
tests. 

Component 2. MISE took the lead in 
developing performance assessments for 
the second component of the plan. In the 
summer of 1999, MISE contracted with 
one of the developers of the performance 
tasks used in the Third International 
Mathematics and Science Study (TIMSS) 
to work with MISE staff and teachers 
from the Partnership districts in a joint 
venture. The intent was to identify avail- 
able performance assessments that could 
be administered across the four Partner- 
ship districts. A working group including 
the consultant, teachers, and MISE staff 
met several times that year to review each 
partner district's curriculum, the avail- 
able performance assessments, and which 
grades should be assessed. This was not 



an easy task. Each district had selected 
their own science modules, and there was 
variation in the modules used in any 
given grade. There also was variation in 
the sequence that was used across the 
schools in a district and within schools. 
Therefore, the working group decided 
that the performance assessments should 
focus on process skills rather than on 
particular content areas to avoid 
advantaging some districts over others. 

The working group reviewed a wide 
range of performance tasks before settling 
on four TIMSS performance assessments: 
two for third-grade students called 
"Magnets and Plasticine," and two for 
seventh-grade students called "Solutions 
and Magnets." The tasks were piloted in 
the fall of 1999 in a sample of classrooms 
across the four districts and then admin- 
istered in all third- and seventh-grade 
classrooms in the spring of 2000. They 
were re-administered with some modifi- 
cations in the spring of 2001 and 2002, 
and additional assessments were selected 
for use in the fourth and eighth grades in 
2002. 

Piloting these assessments and then 
administering them across the four 
districts was an enormous endeavor and 
represented a substantial accomplishment 
by the Partnership. To avoid the biases 
that can come from differential adminis- 
tration, the Partnership developed stan- 
dardized directions for administration. To 
further ensure uniform administration, 
the working group developed a method 
of documenting any assistance provided 
students by the teachers who adminis- 
tered the assessments. The Partnership 
revised the rubrics for scoring the assess- 
ments so that judgments of the quality of 
student work were reliable across mul- 
tiple scorers. 

To maximize the professional devel- 
opment potential of the experience, 
teachers from the Partnership districts 
were involved in most of the develop- 
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merit work for the performance assess- 
ments. At the end of October 1999, the 
Partnership held a multi-day workshop 
for teachers to score student work from 
the pilot, refine the rubrics, and analyze 
the information gleaned from the tasks. 
Teachers first reviewed the rubrics for 
assessing student work, then rolled up 
their sleeves and scored the work. A 
subset of the tasks was scored twice to 
assure inter-rate reliability. Teachers who 
participated in the workshop valued the 
experience; they felt that it was one of the 
most powerful professional activities they 
had ever experienced as teachers. 

In subsequent years, students' work 
on the assessments have been scored 
externally, but teacher leaders have still 
gathered in the fall to examine and 
discuss the results. These teachers have 
led sessions in their districts focusing on 
what the results indicate about the 
strengths and weaknesses of student 
learning in science and the implications 
for curriculum and teaching strategies. 
These teachers have also worked with 
MISE staff to make modifications to the 
assessments and their administration. In 
2001, an experiment was conducted in 
which some third graders were given 
time to examine the materials prior to the 
assessments. 

Components 3 and 4. The demand for 
better informal unit assessments had 
been addressed in part prior to the devel- 
opment of the plan by distributing unit- 
based assessments developed by teachers 
and research scientists from the Educa- 
tional Testing Service. The resulting 
grade-level binders contain more than 
100 tasks and are widely distributed for 
teachers to use with the science modules. 
Shortly after the plan was adopted, 
district teams began to develop and test 
tasks to be used with science modules to 
satisfy the third component. 
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Other factors complicate student 
assessment. The focus of the 
Partnership's science instruction has been 
on inquiry, not recall. It may be that 
students who are learning to question 
and are engaged in inquiry are gaining 
significant skills and understanding that 
the standard assessments do not measure. 
Furthermore, not all of the teachers who 
teach science in grades K-8 have partici- 
pated in the Partnership's professional 
development initiatives, and those who 
have, received varying amounts of train- 
ing. Thus, not all students have access to 
Partnership-trained teachers, and stu- 
dents do not consistently have Partner- 
ship teachers from year to year. This 
makes it harder to measure any long-term 
effect of the use of inquiry-based instruc- 
tion since measures of the average perfor- 
mance on standardized tests like the SAT- 
9 may mask the actual effects of the 
professional development. 

It could also be that the Partnership's 
intervention has had weak effects on 
student performance. This seems unlikely 
given the observed changes in student 
activity and work done in class. Never- 
theless, until adequate measures of 
student achievement are available, these 
questions will remain unresolved. MISE 
staff and district staff have administered 
performance assessments based on the 
TIMSS performance tasks. Taken by all 
students in grades 3 and 7 in the spring of 
2000, 2001, and 2002, the results were 
useful for instructional planning, but did 
not produce reliable student scores. As a 
result, they did not help the Partnership 
solve the larger problem of developing a 
comprehensive assessment of student 
science learning in inquiry-centered 
classrooms. 

Local, State, and National Policies 

MISE is committed to making science 
education a local, state, and national 
priority, and regularly interacts with 
policymakers concurrent with its work in 
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the partner schools. At the local level, 
MISE works with district administrators 
and school board members by presenting 
the Partnership's work at public school 
board meetings or by having informal 
discussions to clarify issues or provide 
more in-depth information. Superinten- 
dents report changes in district policies as 
a result of MISE's influence including 
increased support for strong professional 
development, and improvements in 
hiring and recruitment practices that put 
more emphasis on a teacher's knowledge 
about content and inquiry-based instruc- 
tion. At the state level, MISE staff serve on 
key committees such as New Jersey's 
Professional Teaching Standards Board 
which is responsible for setting high 
standards for teachers' professional 
development. Participation in these 
forums means that science education and 
MISE strategies can influence decisions 
that will impact teachers and students 
statewide. At the national level, MISE staff 
partner with leading educational organi- 
zations such as the National Science 
Resources Center and the Educational 
Testing Service to help develop inquiry- 
centered curriculum and assessment 
tools, and are frequently asked to share 
knowledge about their work with repre- 
sentatives from universities, corporations, 
museums, and school districts around the 
nation. 

Conclusions 

After 10 years of documenting the 
work of the Merck Institute for Science 
Education, CPRE researchers have 
reached the following conclusions about 
MISE's work: 

• The concept of partnership has been 
critical to MISE's success. This idea 
has had a profound impact on MISE's 
approach to reform and its successful 
engagement of the four districts. 
Building a genuine partnership has 
transformed this reform effort from an 
externally funded project to shared 



work based on a common vision. The 
Partnership has sustained and deep- 
ened the reform effort, resulting in its 
integration with the regular opera- 
tions of the four districts. 

• MISE's systemic approach has 
worked. Science has become a priority 
in the partner districts. Inquiry- 
centered teaching has become the 
norm, most teachers have learned to 
use it well, and the districts are 
supporting it. Student work in science 
has changed dramatically and these 
changes have resulted in some im- 
provement on standardized tests. 

• MISE and its partners have not only 
learned how to provide high-quality 
professional development, they have 
learned how to provide it at consider- 
able scale, and they have learned how 
to attract high proportions of teachers 
to participate. 

• The provision of opportunities for 
teachers to acquire new knowledge 
and reflect on their practice, both 
through institutes and on the job, 
matters a great deal. The more profes- 
sional development teachers receive, 
the more their classroom instruction 
resembles the vision of good practice 
advanced by MISE and the better the 
performance of their students. 

• Sustained support for reform pro- 
duces momentum and changes the 
norms of practice. When a critical 
mass of teachers in a school has 
received professional development 
and begun to change their practice, 
the practice of non-participants also 
begins to shift in the same direction. 
When norms change, new teachers 
are socialized and prepared to engage 
in the reformed practice through 
formal induction and informal learn- 
ing from experienced peers. The 
atrophying effects of teacher and 
principal turnover are reduced and it 
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is easier to sustain the new practices 
(CPRE, 2001). 

• Preparing Leader Teachers to act as 
catalysts of change in instructional 
practices across the classrooms of 
their schools produced mixed results 
as it was highly dependent on the 
support of the principal and the 
careful selection of the Leader Teach- 
ers. However, teacher leadership 
proved critical to this reform effort. By 
offering teachers authentic opportuni- 
ties for leadership such as designing 
and delivering professional develop- 
ment, developing and implementing 
new tools of assessment, and leading 
study groups, MISE helped the 
districts build a competent, confident, 
and professional teaching force. The 
districts are able to sustain the work 
in large part due to the capacity of the 
teachers who have emerged as leaders 
through the work of the Partnership. 

• A successful partnership such as this 
one also requires the ongoing support 
of school principals. They set the 
priorities in their schools; shape the 
norms of acceptable practice; deter- 
mine whether teachers have time for 
planning, reflection, and discussion of 
instructional matters; explain the 
changes in classroom practice to 
parents; and find resources and 
provide support for a sustained 
reform. Principals must understand 
and embrace the vision of reformed 
practice and appreciate and support 
the teacher leaders who help spread 
it. 

• The four partner districts have be- 
come active players and shaped the 
Partnership. They have adopted 
MISE's vision of instructional reform, 
and are actively supporting reformed 
practice. District staffs are now more 
attentive to how their policies and 
procedures affect classroom practice. 
The four districts have made changes 



in policy, organization, and assign- 
ments in support of MISE's vision of 
science instruction. They have inter- 
nalized some key lessons drawn from 
this experience and, within their 
resource limitations, are applying 
what they have learned in language 
arts and mathematics. 

• These changes have been given an 
assist by Merck's reputation, exper- 
tise, and commitment to public 
education which have enabled MISE 
to influence state policy and create an 
environment more supportive of the 
reforms. 

• In terms of the bottom line — the 
improvement of student performance 
— classroom observations and inter- 
views show that student work in 
science has shifted dramatically, from 
memorizing facts in textbooks to 
designing and conducting investiga- 
tions that help students understand 
key concepts as well as the habits of 
mind and methods of inquiry used by 
scientists. Analyses of student perfor- 
mance on standardized tests reveal 
that students who have received 
science instruction over several years 
from teachers who have participated 
in the Partnership professional devel- 
opment outperform students who 
have been taught by non-participants. 
In spite of the assessment problems 
encountered by MISE and the Partner- 
ship, these data suggest that, in the 
long run, as more and more teachers 
participate in the workshops, there 
will be an increasing positive impact 
on student performance in science. 

Important Lessons and 
Issues 

The Partnership initiated by MISE has 
been a remarkable collaboration with an 
enviable track record of accomplishment. 
The experience of the Partnership, its 
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successes and shortcomings, and the 
challenges that it has faced offer lessons 
for other organizations that set out to 
help school districts improve teaching. 
Many of these apply to any reform effort. 
The most important lessons are: 

• A shared vision and clear communi- 
cation are critical to the eventual 
success or failure of a reform. MISE's 
advisory committee was a unique and 
important element of the Partnership, 
and helped provide the Partnership 
with valuable feedback to modify and 
enhance the work. The committee 
gave the Partnership's leaders a 
forum to address issues or concerns, 
share ideas, and make decisions 
affecting the work. Providing the 
Partnership's leaders with regular 
opportunities to meet and collaborate 
empowered the key stakeholders and 
gave each a voice in the Partnership's 
development and success. 

• Respect for teachers and teaching is a 
prime directive. From the beginning, 
MISE staff showed respect for teach- 
ers as professionals and valued 
colleagues. They understood that 
teaching was demanding and com- 
plex work, that there were not always 
clear answers to the problems teach- 
ers faced, and that successful teachers 
were highly skilled professionals. 

This respect was evident in their 
voluntary approach to professional 
development, in the climate set in the 
institutes, in their concern about 
quality, and in their insistence that 
accomplished teachers help design 
and lead the professional develop- 
ment. This principle won them the 
respect of the teaching forces in the 
four districts. Even those who were 
unwilling to participate recognized 
that the "MISE folks understood 
teaching." 

• If you build good professional devel- 
opment programs, teachers will come. 



CPRE's data show that the vast 
majority of teachers will voluntarily 
take advantage of opportunities to 
learn and to improve their teaching 
practice — if the opportunities are 
seen as worthwhile and are accessible. 
Offering professional development 
that is directly related to teachers' 
work, showing respect for their 
professionalism, carefully combining 
attention to subject-matter knowledge 
with skills in pedagogy, and class- 
room management of inquiry have 
proved to be a successful formula. 

Professional development must be a 
continuous process, not a set of 
episodes. Not only do teachers benefit 
from continued engagement with the 
content of the curriculum and oppor- 
tunities to reflect on what students are 
learning, there is also constant turn- 
over. Good teachers who have been 
trained depart, and each year there is 
an influx of new teachers who have to 
be brought up to par quickly. The 
reassignment of teachers to different 
grade levels after they have devel- 
oped expertise with the curriculum at 
a given grade level also confounds 
systematic staff development plan- 
ning. MISE and the partner districts 
have responded by building pro- 
grams to introduce new teachers to 
inquiry and in which all new teachers 
are required to participate. These 
orientation programs are effective for 
introducing new staff to district 
instructional expectations, but consid- 
erable time is required for high levels 
of instructional proficiency. 

Given the current policies governing 
the preparation and assignment of 
elementary school teachers, there are 
limits on the changes that can be 
made in science teaching even with 
good professional development. Most 
elementary school teachers have little 
content knowledge in science to begin 
with, so it is difficult for them to reach 
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the level of content knowledge 
needed for successful inquiry-cen- 
tered instruction. Many can only 
move part of the way toward re- 
formed practice. They need the 
structure provided by the well- 
designed science modules and the 
structured investigations that are built 
into them. In addition, science com- 
petes with literacy and mathematics 
for instructional time in elementary 
school classrooms. Teachers may not 
be able to do inquiry every day 
because it takes considerable time to 
plan and carry out. 

Persistence is critical as it takes time 
for teachers to acquire the knowledge, 
skills, and confidence to change their 
practices, and even longer to alter the 
norms of practice. MISE has worked 
with the four partner districts for a 
decade. One of the important roles 
that they played was to keep people 
focused on the reform. Many compet- 
ing interests pulled at people's time 
and attention. Keeping the interest of 
the reform at the top of the agenda is 
not an easy task and takes skill, 
patience, and tenacity. 

Personnel changes are common in 
school districts, and this turnover is 
the enemy of persistence and, there- 
fore, of success. Superintendents, 
school boards, curriculum supervi- 
sors, principals, and teacher leaders 
come and go. Since 1993, MISE has 
worked with 14 superintendents in 
the four districts. The annual turnover 
of teachers was about 6%. However, 
the broad participation in the work 
and the widespread respect that it 
garnered made it possible to sustain 
the reforms in spite of the turnover in 
personnel. MISE staff kept top leader- 
ship informed and educated along the 
way and garnered their support and 
approval. They built support for the 
effort among all of the stakeholders so 
that the loss of one leader did not 



bring the effort to a halt. This has 
been a time-consuming and delicate 
process, but also an important role 
that MISE was well positioned to 
play. Merck's reputation for quality 
and integrity have been important in 
sustaining the work of the Partner- 
ship through numerous leadership 
transitions. 

• Teachers' knowledge and skills are 
critical factors in the classroom learn- 
ing experience, but not the only ones. 
Good curriculum materials are also 
essential. Teachers need access to and 
support in implementing standards- 
based curricula and teaching materi- 
als. They need the support and 
knowledgeable involvement of 
school- and district-level administra- 
tors, parents, and the community. 
MISE addresses these needs through 
resource centers featuring exemplary 
science education materials, Merck 
employee volunteer programs, and 
parent involvement programs, in 
addition to its support of long-term 
professional development. 

• Better assessment tools in science are 
needed. Existing measures do not 
adequately show the effects of better 
science instruction. Right now, teach- 
ers see improved student work in 
their classrooms and a higher level of 
student interest in science, but the 
available measures do not adequately 
demonstrate this change to parents, 
school leaders, or the public. In 
addition to assessments that provide 
good diagnostic information for 
teachers' instructional planning, 
assessments that are persuasive to the 
public and policymakers are needed 
as well. 

• The state policy context on incentives 
for change can play a pivotal role in 
stimulating instructional reform, and 
MISE's role in shaping state policy 
has had a high payoff. Including 
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science in state assessments is particu- 
larly important. If science is not 
assessed, attention shifts to the as- 
sessed subjects and it becomes diffi- 
cult to engage teachers in serious 
instructional reforms. This problem 
can be countered if district leaders 
make science a priority and include it 
in internal accountability processes. 

Final Thoughts 

MISE is a compelling example of a 
technical assistance organization that has 
developed a coherent approach to reform 
and paid careful attention to feedback 
from the field and from evaluation to 
modify its strategy as needed. The suc- 
cess of the Partnership is testimony to 
MISE's thoughtful strategy, patient 
responsiveness, and willingness to make 
changes. Each year, CPRE evaluators 
reviewed their findings with MISE staff 
who then used the information to alter 
the course of their work. MISE is a learn- 
ing organization, and it has modified its 
tactics as needed. Learning that the 
Leader Teacher strategy was not effec- 
tively spreading good practice, MISE 
altered its approach to professional 
development to engage hundreds of 
teachers through Peer Teacher Work- 
shops. Finding that existing curricular 
guidance and assessment tools were 
inadequate, MISE staff developed local 
curriculum frameworks and new assess- 
ments. Recognizing that state policies 
were limiting the reforms in the districts, 
MISE staff actively sought to create a 
more supportive policy environment, 
participating in the development of state 
standards and curriculum frameworks, 
engaging in public outreach, and net- 
working with other business-school 
partnerships, thereby broadening the 
base for reform. 

MISE formed partnerships with the 
four districts that were based on mutual 
respect, frankness, and honesty, and they 
blossomed into one common Partnership 



with a shared vision of reform. This 
increased the likelihood that the collec- 
tive effort would be sustained over time; 
since everyone was a player in the deci- 
sion-making process, everyone had a 
stake in the results. 

MISE understood that a systemic 
approach was needed in order for this 
initiative to have the desired impact of 
bringing real change to science instruc- 
tion. Without a systemic approach, it is 
likely that the Partnership would have 
been less effective. By working at every 
level with school districts, by being 
mindful of the political nuances of each 
setting, and by addressing issues of time 
and professional development, MISE 
increased the effectiveness of its efforts. 

The MISE Partnership, in addition to 
being a story of real accomplishments, is 
an incomplete story. It is an incomplete 
story because as yet, there are not the 
broad-based student achievement results 
needed to support the assertions of 
teachers, administrators, MISE staff, and 
even CPRE evaluators that students are 
learning more science. This is a critical 
and important point that is being ad- 
dressed by MISE but which leaves an 
empty space in this success story. 

With all of the intensive, high-quality 
professional development that was part 
of the Merck venture, it is still sobering to 
realize that most teachers need between 
80 and 100 hours of professional develop- 
ment to produce significant change in 
instruction. The extension of this fact to 
other districts in the two states under- 
scores the need for enormous funding 
support and patience in order to see the 
changes in practice needed to ensure that 
all children receive a sound science 
education. 

This is a partnership worthy of mod- 
eling. The example is there not only for 
other corporations to follow, but for all 
technical assistance organizations 
whether they be sponsored by states. 
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universities, non-profit organizations, or 
professional organizations. There are 
similar stories to be told about the El Paso 
Collaborative for Academic Excellence, 
the Bay Area School Reform Collabora- 
tive, the Southern Maine Collaborative, 
and other organizations that have fo- 
cused on particular communities or 
regions and provided the intense and 
sustained assistance needed to make 
significant changes in teaching. After 
almost a decade of financial support, 
sustained professional development, and 
management guidance, the four school 
districts working with MISE have made 
dramatic changes to their K-8 science 
programs. Children have access to better 
science materials and teachers are more 
comfortable with their teaching. The 
success of MISE and the Partnership 
demonstrates that serious efforts to help 
teachers improve their practice brought 
positive results, and that students ben- 
efited from these investments. 
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